Virtual presence device, system, and method

ABSTRACT

An example virtual presence system includes at least a first and second mobile device in electronic communication with one another, a panoramic camera coupled to the second mobile device. A user of the first mobile device can specify a position (e.g., location or orientation), such as via a joystick or head-mounted display. The specified position is transmitted to the second mobile device, which selectively obtains image data from the panoramic camera corresponding to the specified position. The selected image data is transmitted as a video stream or other format to the first mobile device, thereby allowing the user of the first mobile device to view surrounding environment available at the second mobile device.

PRIORITY CLAIM

This application is a continuation of U.S. patent application Ser. No.15/662,016, filed on Jul. 27, 2017, entitled “VIRTUAL PRESENCE DEVICE,SYSTEM, AND METHOD;” which claims the benefit of priority from U.S.Provisional Patent Application No. 62/367,456, filed on Jul. 27, 2016,entitled “VIRTUAL PRESENCE DEVICE, SYSTEM, AND METHOD,” the contents ofwhich are incorporated herein by reference in their entireties.

TECHNICAL FIELD

The present disclosure relates to remote two- or multi-way videoparticipation devices, and more particularly, to a remote virtualpresence device allowing remote control of 360-degree viewing ofdestination surroundings and transfer of data between origin anddestination surroundings, for example in substantially real time.

BACKGROUND

There is an increasing demand for expediting communication andinformation transfer in all aspects of modern life, including business,social, and private settings. Among such demands is the desire and theneed to remotely participate in a distant environment without beingphysically present.

There are several apps and software systems currently available such asGoogle® Hangouts®, Apple® Facetime®, WhatsApp®, Viber®, and Skype® thatallows a person to communicate and view another location in real-timewith the use of devices such as smartphones that have wirelesscapabilities, built-in cameras and voice input and output facilitiessuch as microphones, speakers and headphones.

Present real-time communication systems however do not allow the usersthe freedom to view in the direction they desire. Their view is insteaddependent upon the direction where the other device is being pointed toby the other end user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a virtual presence system according to oneembodiment.

FIG. 2 is a schematic view of a virtual presence system according toanother embodiment.

FIG. 3 is an isometric view of a mobile device including a 360 degreeimage capture device according to one embodiment.

FIG. 4A is an isometric view of a mobile device including a 360-degreeimage capture device according to another embodiment.

FIG. 4B is a partial side view of the mobile device of FIG. 4A accordingto one embodiment.

FIG. 5 is an isometric view of a mobile device including a 360-degreeimage capture device according to yet another embodiment.

FIG. 6 is an isometric view of a multidimensional capture and displaydevice according to one embodiment.

FIG. 7 is an isometric view of a multidimensional capture and displaydevice according to one embodiment.

FIGS. 8A-8B are example flow diagrams of virtual presence processesperformed by example embodiments.

FIG. 9 is a block diagram of an example computing system forimplementing elements of a virtual presence system according to anexample embodiment.

DETAILED DESCRIPTION

Embodiments described herein provide enhanced computer- andnetwork-based methods, devices, and systems for remote virtual presence.Some embodiments provide a remote virtual presence device allowingremote control of 360-degree viewing of remote surroundings insubstantially real time. The remote virtual presence device may bemobile, such as a smart phone coupled to a 360-degree camera.Alternatively, the remote virtual presence device may be substantiallyfixed in location, such as a conferencing device located in an officesetting or a capture device mounted to a fixed structure (e.g., on autility pole, on a building, in a sports arena, in a concert venue). Theremote virtual presence device may also or instead be mounted to avehicle, such as a car, boat, mobile robot, aerial piloted or unpilotedvehicle (e.g. drone), or the like.

1. Overview

Through this disclosure various methods, systems, and devices aredescribed to provide a thorough understanding of certain embodiments. Itis understood however that other embodiments may include varyingcombinations of the described and not described methods and systemfeatures suitable to enable transmission and reception of live, orsubstantially live, streaming audio, visual and motion data betweendevices, allowing end users to communicate verbally with the ability toselectively view remote surroundings by controlling the directional viewof a panoramic camera that responds to a three dimensional movement of aheadset, joystick, or a suitable pointing device. All such variationsare contemplated to be within the scope of the present disclosure. Forclarity of description wherever electronic communication is referred toin this disclosure, the phrase includes any form of wireless, wired,electronic, Bluetooth, other protocols, and/or any suitable protocol forinformation communication between devices.

FIG. 1 illustrates a possible configuration of a virtual presence system100 according to one embodiment. According to one aspect, the system 100includes a first mobile device 150 and a second mobile device 160remotely coupled to one another via one or more cellular (e.g., 4G LTE)and/or wireless protocols for communication (e.g., WiFi 802.11, WiMAX802.16), or other suitable means such as Bluetooth.

The system 100 can include a viewing device 107, which can be worn by auser 106, and which is in wireless communication 108 (e.g., Bluetooth)with the first mobile device 150. In typical embodiments the firstmobile device 150 is a smart phone, but it could instead by a tabletcomputer, a laptop computer, or the like. The viewing device 107 in thisexample is a headset display, such as virtual reality goggles, eyewear,helmet, or other head-mounted display. The first mobile device 150 canin one aspect include at least one camera 102, a viewing screen 103, amicrophone 104, and a speaker 105.

According to one aspect, the second mobile device 160 includes a360-degree image capture device 140 coupled to a body thereof. In thisaspect, the capture device 140 can include a camera and/or any othersuitable motion image or image-stitching device or a deviceincorporating a motion image or image-stitching application. Forclarity, and without any intention to limit the scope of the presentdisclosure the capture device will hereinafter be referred to as camera140.

The camera 140 can be rotatably coupled to, or with respect to, the body123 of the second mobile device 160. The system 100 in one embodimentcan be configured to facilitate controlling a direction of view from thecamera 140 situated on the second smartphone 160 by head movement of theuser 106 while wearing the viewing device 107.

In the illustrated embodiment, the viewing device 107 includes a monitoror screen to communicate images to the user 106. The displayed videoimages are obtained from the first mobile device 150, which receivesthem from the second mobile device 160, which in turn obtains them fromthe camera 140.

Embodiments provide a communication protocol that facilitatescommunication and interaction between the illustrated devices. Forexample, in one embodiment, the viewing device 107 is configured totransmit movements and position (e.g., head movement) of the user 106along a three-dimensional axis 109 to the first mobile device 150. Thefirst mobile device 150 includes an application, program, or similarsoftware module that transmits data relating to movement, audio, and/orvisual data to the second mobile device 140. Such transmission can occurdirectly or indirectly between the first mobile device 150 and thesecond mobile device 160.

Upon receiving information about the head movements and/or position ofthe user 106, an application, program, or other software module on thesecond mobile device 160 in turn determines a directional view of thecamera 140 for the user 106. For example, if the user 106 tilts his headupwards, this action (or at least the new location) of the user's headis transmitted by the first mobile device 150 to the second mobiledevice 160. The second mobile device 160, in cooperation with the camera140, obtains one or more relevant images, such as by preparing and/orobtaining one or more individual images or an encoded video stream(e.g., MPEG stream) from the camera 140. The obtained image data is thentransmitted by the second mobile device 160 back to the first mobiledevice 150.

In one embodiment, the system 100 can include one, or a network of,first cellular towers 111. In one embodiment, the first mobile device150 can include an interim wireless transmission features to initiate aninterim wireless transmission 110 of material such as live, orsubstantially live, stream audio, visual, and motion and/or positiondata to the first network of cell towers 111 (collectively, “ClientData”).

In one embodiment, the system 100 can include one, or a network of,Internet provider, or equivalent, systems 113. In one embodiment, thefirst cell towers 111 can communicate the Client Data through wirelesstransmission 112 to the Internet provider system 113. In one aspect, thesystem 100 can include a service provider 114, which can manage theInternet provider system 113.

In one aspect, the Internet provider system 113 can transmit the ClientData via wireless or cellular or other electronic communication 115directly or indirectly to the second mobile device 160. In one aspect,the Internet provider system 113 can transmit the Client Data viawireless or cellular communication or other electronic communication 115to a second network of cellular towers 116, which in turn is configuredto locate and wirelessly or through cellular or other electroniccommunication 117 transmit the Client Data to the second mobile device160.

In one aspect, the camera 140 of the second mobile device 160 includes aplurality of viewing features, lenses, and/or capturing mediums 118(e.g., CMOS or CCD sensors/chips). For clarity and without any intentionto limit the scope of the present disclosure, such viewing feature ishereinafter referred to as lens(es) 118. In one aspect, the secondmobile device 160 can include another image capture device or standardcamera or recording device 119, a microphone 120, a speaker 121 and aviewing screen or monitor 122.

In one embodiment, the camera 140 can transmit and/or electronicallycommunicate images and/or motion images or pictures from surroundingsthereof, as well as audio data (collectively, “Host Data”) to the firstmobile device 150 and/or the viewing device 107 in a reverse fashion asthat described with respect to Client Data electronic communication ortransmission from the first mobile device 150.

Therefore, according to one embodiment, Client Data can be, throughwireless and/or cellular, or electronic communication, or other remotedata transfer protocols, for example Bluetooth communication for shorterranges, transferred to the second mobile device 160, therebymanipulating the camera 140. For example, motion of the user 106 headand/or body, imparting motion to the viewing device 107 can in oneaspect manipulate and/or direct proportionally and directionally, thecamera 140. In such an example, the camera 140 then transmits to thescreen or monitor of the viewing device images and/or motion images orpictures of the corresponding surroundings thereof along the axescoordinates associated with the user 106 motion. Some embodimentsincorporate retina movement detection techniques, in order tocommunicate motion stemming from movement of the retina or eyeball ofthe user 106.

The described techniques provide important advances over the prior art.In particular, by automatically accounting for and transmittinginformation about changes in position and movement at the client end(e.g., viewing device 107 and/or mobile device 150), there is no needfor the user 106 to instruct the remote device 160 to change the cameraangle or focus on a different subject. If the user 106 wants to look tothe right, he need only turn his head to the right. The new positioninformation will be transmitted to the remote device 160, which willobtain the corresponding image data from the camera 140.

Note also that the remote device 160 tracks and compensates for at leastsome changes in position and orientation of the remote device 160 and/orthe camera 140 itself. It does so in order to provide the user 106 withthe appearance of a fixed frame of reference. The software moduleexecuting on the remote device 160 subtracts out any movement of theremote device 160. For example, if the mobile device 160 is turned 90degrees to the right, but the user 106 has not moved, then the user 106is provided with the same field of view as before the movement of thedevice 160, which would require obtaining image data from a differentportion of the camera 140 field of view, specifically 90 degrees left ofthe previous view in order to compensate for the rightward turn of thedevice 160.

Note that changes in the movement (a change of position as opposed toorientation) of the remote device 160 may not be compensated for by thedevice 160. Some changes in position may result in only minor changesthe view of the camera 140. For example, if the subject is a far offlandscape, moving the device 160 a meter or two in any direction willnot significantly change the view of the camera 140. But for closer-insubjects, even minor positional changes can result in significantchanges to the camera 140 view. In such cases, the device 160 may, basedat least in part on whether the view has changed by more than athreshold amount, determine the relevant positional changes and transmitthose to the device 150, such that some indication of movement can begiven to the user 106. For example, an icon, sound, or other messageindicating movement may be presented on the viewing device 107, so as toreduce disorientation by the user 106.

In some embodiments, the remote device 160 may also incorporate (or beconnected to) multiple directional microphones, arranged to provide a360-degree soundscape. Based on the orientation of the viewing device107, different audio (e.g., different microphones or mixtures thereof)may be selected for transmission to the device 150. As the user 106reorients his head, he will perceive changes in the audio consistentwith rotation or other corresponding changes in position at the remotelocation.

In some embodiments, a system management service (not shown) may beaccessible via the network 113. The service tracks devices that are partof the system 100, in order to provide search and discovery services, inaddition to ancillary services, such as user account management, paymentprocessing, and the like. Using this service, the user 106 may finddevices (e.g., device 160) based on location, venue, activity, or thelike, and then begin to interact with those devices in order to obtainvideo and/or audio data.

The present disclosure sets forth examples of a system of devices andsystems facilitating two or more users to communicate, and otherembodiments and/or system can be combined or assembled from variousfeatures of the described embodiments. For example, a plurality of usersor mobile devices can be replaced with a tablet or laptop or televisionsor multi-screen communication devices. In addition, various 360-degreescameras can be used, for example they can either be integrally builtinto a mobile device or they can be mounted onto the mobile device as aseparate device. In addition the at least two communication devices cantransmit Client and Host Data back and forth by other means ofcommunication network.

FIG. 2 illustrates a virtual presence system 200 illustrating a laptop250 and a display and capture device 260 having a plurality of viewingscreens and/or monitors that can communicate via wireless and/orcellular or other electronic communication.

In one aspect, a user 206 is able to control a direction of view fromthe 360-degree camera 240 coupled on the display and capture device 260via operating a joystick 207. The joystick 207 can be configured,adapted, and/or operable to transmit via electronic communication 208,the user's joystick input along a three-dimensional axis 209 to thelaptop 250 having an application or software which transmits thejoystick 207 movement data in addition to audio and visual data(collectively, “Client Data”) to the display and capture device 260thereby controlling the directional view of the camera 240 for the user206 to transmit images from the camera 240 to the laptop 250 in similarfashion to that described in relation to the previous embodimentsrelated to FIG. 1.

In one embodiment, the laptop 250 includes at least a camera 202,viewing screen or monitor 203, a microphone 204, and a speaker 205configured to transmit through wireless, cellular, and/or Bluetooth orother electronic communication 210 the Client Data to a network of celltowers 211 which in turn forwards wirelessly 212 to an Internetprovider, or equivalent, network 213. In one aspect, a service provider214 manages the network 213 (e.g., and Internet Protocol network).

The network 213 then in one embodiment can be configured to transmit theClient Data via electronic communication 215 to another network of celltowers 216 that locates and transmits through electronic communication217 the data to the image and capture device 260. In one aspect, thecamera 240 can include a plurality of viewing features, lenses, and/orcapturing mediums 218. For clarity, and without any intention to limitthe scope of the present disclosure such viewing feature is hereinafterreferred to as lens(es) 218. In one aspect, the image and capture devicecan include a microphone 220, a speaker 221, and at least one viewingscreen 222. In this FIG. 2, the communicating device 260 has fourviewing screens, two of which are not shown in this view and are presenton the rear side of the device 260. However, in other embodiments it caninclude a greater or lesser number of screens or be comprised of a shapeother than a cubical shape.

FIG. 3 illustrates an isometric view of a mobile device 360 including atleast one integrated 360-degree camera system 340 coupled thereto andhaving a plurality of lenses 318, a standard camera 319, a microphone320, a speaker 321 and a viewing screen 322. In one aspect, the camerasystem 340 can capture images from more than one-dimensional coordinatessimultaneously and be rotatably coupled to, or with respect to, a bodyof the mobile device 360. In one aspect, the mobile device 360 andcamera system 340 can operate and/or be part of a virtual presencesystem as described with respect to the second mobile device 160 in theembodiment of FIG. 1.

FIG. 4A illustrates an isometric view of a mobile device or tablet 460,and FIG. 4B a partial side view thereof, according to one embodimentthat includes at least an integrated and/or coupled thereto panoramic or360-degree camera system 440 having a plurality of lenses 418, astandard camera 419, a microphone 420, a speaker 421, and a viewingscreen 422. In one aspect, the camera system 440 is embedded within aportion of a body of the mobile device 460, for example toward aperiphery thereof, such as an upper periphery portion.

FIG. 5 illustrates an isometric view of a mobile device or tablet 560according to one embodiment and a 360-degree and/or panoramic camerasystem 540 having a plurality of lenses 518 and coupling device 524,such as a jack, attached thereto. The mobile device 560 in oneembodiment includes at least one standard camera 519, a microphone 520,a speaker 521 and a viewing screen 522. According to this embodiment,the camera system 540 can be removably coupled to the mobile device 560via the coupling device or jack 524 and the input/output coupling slot523 incorporated in the mobile device 560. The coupling device or jack524 can include a ⅛″ video jack in some embodiments. The camera system540 can in one aspect be configured to be removed from the mobile device560 and coupled to another device, tablet, mobile device, and/or otherimage capture and display device.

FIG. 6 illustrates a display and capture device 660 having at least one360-degree and/or panoramic camera system 640 having a plurality oflenses 618, a microphone 620, a speaker 621, and a at least two, or aplurality of, viewing screens, monitors, and/or displays 622. Themonitors 622 can in one embodiment display live stream videoconferencing in a situation where there are several persons on one endof the communication such as a business meeting situation. The displayand capture device 660 in one example application allows persons in ameeting to view the in-caller who wishes to attend the meeting invirtual live streaming. The camera system 640 can in one aspect beintegrated to a body of the display and capture device 660, or removablein another aspect, and it allows the in-caller to freely view around themeeting room through the use of a viewing apparatus 107 in FIG. 1 or bymeans of a joystick 207 and a laptop 250 in FIG. 2, as is described withrespect to the embodiments of these figures, or not describedembodiments equivalent thereto.

In some embodiments, the device 660 includes an acoustic sourcelocalization module. In such embodiments, an array of two or moremicrophones is used to determine the origin of a sound (e.g., aspeaker's voice) via triangulation or time difference of arrival. In avideo conferencing setting, the device 660 may be placed at the middleof the table, surrounded by multiple conferees. The acoustic sourcelocalization module can be used to determine the location of thespeaker, which can be used to determine a portion of the field of viewof the camera 640 from which to transmit image or video data back to thelaptop 250, device 150, or other remote system/device. Image data fromthe determined portion of the camera field of view may be transmittedconcurrently with image data from the camera field of view that reflectsthe selected view of the remote user.

FIG. 7 illustrates a display and capture device 760 having at least one360-degree and/or panoramic camera system 740 having a plurality oflenses 718, a microphone 720, a speaker 721, and a at least two, or aplurality of, viewing screens, monitors, and/or displays 722. Themonitors 722 can in one embodiment display live stream videoconferencing in a situation where there are several persons on one endof the communication such as a business meeting situation. The displayand capture device 760 in one example application allows persons in ameeting to view the in-caller who wishes to attend the meeting invirtual live streaming. The camera system 740 can in one aspect beintegrated to a body of the display and capture device 760, or removablein another aspect, and it allows the in-caller to freely view around themeeting room through the use of a viewing apparatus 107 in FIG. 1 or bymeans of a joystick 207 and a laptop 250 in FIG. 2, as is described withrespect to the embodiments of these figures, or not describedembodiments equivalent thereto.

In the illustrated embodiment of FIG. 7, the camera system 740 iscoupled to the body of the display and capture device 760 via a stand723 configured to space the camera system 740 from the body of thedisplay and capture device 760 and better capture a 360 degree viewabout the display and capture device 760, including an area below thehorizontal plane. In one embodiment, the display and capture device 760can include a shroud or cover 724 for ornamental or functional purposes.

In one embodiment, the camera system 740 is configured to be inwireless, wire, Bluetooth, or other electronic communication with themonitors 722.

It is understood that various embodiments may include various featuresor combination of the various equipment, without departing from thescope of the present disclosure. For example, FIG. 1 both mobile devices150 and 160 can include either an integrated camera system 140 or aremovable camera system 540 in FIG. 5 can be mounted onto the mobiledevices 150 and 160. In addition or instead, a tablet, smartphone,and/or any other known or to be introduced mobile device, can be used asthe mobile device. Furthermore both end users can utilize a viewingapparatus 107 in addition to the communication system 100 show inFIG. 1. Another example in FIG. 2, the communication system 200 caninclude either the use of laptops 250 and joysticks 207 at both ends.

Alternatively, the communication system 200 can include multi-screencommunication device system 260 at both ends to allow groups or peopleto communicate coupled with the use of a suitable pointing device suchas a joystick 207.

Some embodiments provide a virtual venue attendance function. Publicvenues, such as arenas, stadiums, concert halls, and the like can bemodified to incorporate multiple host devices which function as virtual“seats.” Remote users can then use the techniques described herein toexperience events held at a venue. By using a joystick or viewinggoggles, the user can obtain different views of the event.

Therefore, at least certain embodiments of the present disclosure,whether described or not described, allow the end users to communicateand view freely in any direction they desire without having to rely onthe other person to direct the camera in any specific direction.

2. Example Processes

FIGS. 8A-8B are example flow diagrams of virtual presence processesperformed by example embodiments. By way of overview, FIG. 8Aillustrates a client process performed by a module executing on a clientdevice, such as device 150. FIG. 8B illustrates a host process performedby a module executing on a remote host device, such as device 160.

FIG. 8A illustrates a client process that begins at block 802, where itreceives position data. The position data may be received from an inputdevice, such as a joystick or head-mounted display (e.g., goggles). Theposition data may describe location (in two or three dimensions),orientation, and/or movement (e.g., velocity, acceleration).

At block 804, the process transmits the position data to a remote hostmodule, such as may be executing on device 160. The process may alsotransmit video and/or audio obtained by the module, such as from acamera on device 150.

At block 806, the process receives video and/or audio data from theremote host module. As described herein, the video/audio data may beselected by the remote host module/device based on the position dataprovided by this process. The process may receive other or additionaldata from the remote host module, such as position (e.g., movement) dataabout the remote device, or the like.

At block 808, the process presents the received video and/or audio datato a user. For example, the process may transmit the received data toviewing goggles, display data on an incorporated display device, or thelike.

At block 810, the process continues to block 802 unless it hasdetermined to stop, such as because a user as instructed it to do so.

FIG. 8B illustrates a host process that begins at block 852, where itreceives position data from a remote client module. The remote clientmodule may execute a process such as that described with respect to FIG.8A, above. As discussed there, the position data may be received from aninput device, such as a joystick or head-mounted display (e.g., eyewear,goggles, in-helmet display) in communication with the client device.

At block 854, the process obtains video and/or audio data based on thereceived position data. Obtaining video/audio may include selecting theappropriate video data from a field of view provided by a panoramiccamera, such as camera 140 in FIG. 1. The particular video data selectedmay be determined based on the receive position data, such that thevideo reflects changes in the position of the remote user. The videodata may further be selected based on position data that describes thelocation, orientation, or movement of the host device and/or camera. Inparticular, some embodiments may compensate for movement of the hostdevice, in order to provide a consistent frame of reference for the userof the client device, such as by adjusting the selected camera view tocompensate for any movement of the camera.

At block 856, the process transmits obtained video and/or audio data tothe remote client module. Other data may be transmitted, such asposition (e.g., movement) data that reflects the motion of the hostdevice that is executing this process.

At block 858, the process continues to block 852 unless it hasdetermined to stop, such as because a user as instructed it to do so.

Although the blocks in the above flow diagrams are shown as executing ina particular sequence, other embodiments may perform the blocks in otherorders or at least in part concurrently.

3. Example Computing System Implementation

FIG. 9 is a block diagram of an example computing system forimplementing functions of a virtual presence system 100 according to anexample embodiment. In particular, FIG. 9 shows a computing system ordevice 10 that may be utilized to implement a client/host manager 50.The manager 50 may be part of a client device, such as mobile device 150or a “host” device, such as remote mobile device 160 in FIG. 1. Themanager 50 performs one or more of the process described above.

Note that one or more general purpose or special purpose computingsystems/devices may be used to implement the described techniques.However, just because it is possible to implement the manager on ageneral purpose computing system does not mean that the techniquesthemselves or the operations required to implement the techniques areconventional or well known. In addition, the computing system 10 maycomprise one or more distinct computing systems/devices and may spandistributed locations. Furthermore, each block shown may represent oneor more such blocks as appropriate to a specific embodiment or may becombined with other blocks. Also, the manager 50 may be implemented insoftware, hardware, firmware, or in some combination to achieve thecapabilities described herein.

In the embodiment shown, computing system 10 comprises a computer memory(“memory”) 11, a display 12, one or more Central Processing Units(“CPU”) 13, Input/Output devices 14 (e.g., keyboard, mouse, CRT or LCDdisplay, and the like), other computer-readable media 15, and a networkconnection 16. The I/O devices 14 may include or provide an interface toa camera 17 (e.g., camera 140), a viewing device 18 (e.g., goggles 107),and/or other input/output devices illustrated and described above.

The manager 50 is shown residing in memory 11. In other embodiments,some portion of the contents, some or all of the components of themanager 50 may be stored on and/or transmitted over the othercomputer-readable media 15. The manager 50 preferably executes on one ormore CPUs 13 and performs the techniques described herein. Other code orprograms 30 (e.g., operating system, application programs, etc.) andpotentially other data repositories, such as data repository 20, alsoreside in the memory 11, and preferably execute on one or more CPUs 13.Of note, one or more of the components in FIG. 9 may not be present inany specific implementation. For example, some embodiments may notprovide other computer readable media 15 or a display 12.

The manager 50 interacts via the network 99 with other devices 10′ and10″. For example, in the context of FIG. 1, if device 10 is the clientdevice 150, then devices 10′ and 10″ may be two instance of the remotedevice 160. The network 99 may be any combination of media (e.g.,twisted pair, coaxial, fiber optic, radio frequency), hardware (e.g.,routers, switches, repeaters, transceivers), and protocols (e.g.,TCP/IP, UDP, Ethernet, Wi-Fi, WiMAX) that facilitate communicationbetween remotely situated humans and/or devices. The other devices 10′and 10″ are constituted similarly to device 10. The devices 10′ and 10″may be personal computing devices, such as desktop computers, laptopcomputers, smart phones, tablets, or the like.

Also included in the memory 11 are a user interface manager 41 and anapplication program interface (“API”) 42. The UI manager 41 provides aview and a controller that facilitate user interaction with the manager50 and its various components. For example, the UI manager 41 mayprovide interactive access to the manager 50, such that users caninteract with the manager 50, such as by searching for and/or selectinga remote device with which to interact, changing display settings, orthe like. In some embodiments, a central server (not shown) tracksdevices that are part of the system 100, in order to provide search anddiscovery services, in addition to ancillary services, such as useraccount management, payment processing, and the like. Such a server maybe connected to the network 99 and accessible via the UI manager 40.

The API 42 provides programmatic access to one or more functions of themanager 50. For example, the API 42 may provide a programmatic interfaceto one or more functions of the manager 50 that may be invoked by one ofthe other programs 30 or some other module. In this manner, the API 42facilitates the development of third-party software, such as userinterfaces, plug-ins, adapters (e.g., for integrating functions of themanager 50 into Web applications), and the like.

In addition, the API 42 may be in at least some embodiments invoked orotherwise accessed via remote entities, such as code executing on one ofthe network devices 10′ and 10″, to access various functions of themanager 50. For example, Client and/or Host Data may be obtained from orprovided to the manager 50 via the API 42. The API 42 may also beconfigured to provide management widgets (e.g., code modules) that canbe integrated into third-party applications and that are configured tointeract with the manager 50 to make at least some of the describedfunctionality available within the context of other applications (e.g.,mobile apps).

In an example embodiment, components/modules of the manager 50 areimplemented using software programming techniques. For example, themanager 50 may be implemented as a “native” executable running on theCPU 13, along with one or more static or dynamic libraries. In otherembodiments, the manager 50 may be implemented as instructions processedby a virtual machine that executes as one of the other programs 30. Ingeneral, a range of programming languages may be employed forimplementing such example embodiments, including representativeimplementations of various programming language paradigms, including butnot limited to, object-oriented (e.g., Java, C++, C#, Visual Basic.NET,Smalltalk, and the like), functional (e.g., ML, Lisp, Scheme, and thelike), procedural (e.g., C, Pascal, Ada, Modula, and the like),scripting (e.g., Perl, Ruby, Python, JavaScript, VBScript, and thelike), and declarative (e.g., SQL, Prolog, and the like).

The various components may be implemented using more monolithicprogramming techniques, for example, as an executable running on asingle CPU computer system, or alternatively decomposed using a varietyof structuring techniques, including but not limited to,multiprogramming, multithreading, client-server, or peer-to-peer,running on one or more computer systems each having one or more CPUs.Some embodiments may execute concurrently and asynchronously, andcommunicate using message passing, remote procedure call, or otherdistributed computing paradigms. Equivalent synchronous embodiments arealso supported. Also, other functions could be implemented and/orperformed by each component/module, and in different orders, and bydifferent components/modules, yet still achieve the described functions.

In addition, programming interfaces to the data stored as part of themanager 50, such as in the data store 20, can be available bylanguage-specific APIs; libraries for accessing files, databases, orother data repositories; through representational languages such as XML;or through Web servers, FTP servers, or other types of servers providingaccess to stored data. The data store 20 may be implemented as one ormore database systems, file systems, or any other technique for storingsuch information, or any combination of the above, includingimplementations using distributed computing techniques.

Different configurations and locations of programs and data arecontemplated for use with techniques described herein. A variety ofdistributed computing techniques are appropriate for implementing thecomponents of the illustrated embodiments in a distributed mannerincluding but not limited to TCP/IP sockets, RPC, RMI, HTTP, WebServices (XML-RPC, JAX-RPC, SOAP, and the like). Other variations arepossible. Also, other functionality could be provided by eachcomponent/module, or existing functionality could be distributed amongstthe components/modules in different ways, yet still achieve thefunctions described herein.

Furthermore, in some embodiments, some or all of the components of themanager 50 may be implemented or provided in other manners, such as atleast partially in firmware and/or hardware, including, but not limitedto one or more application-specific integrated circuits (“ASICs”),standard integrated circuits, controllers executing appropriateinstructions, and including microcontrollers and/or embeddedcontrollers, field-programmable gate arrays (“FPGAs”), complexprogrammable logic devices (“CPLDs”), and the like. Some or all of thesystem components and/or data structures may also be stored as contents(e.g., as executable or other machine-readable software instructions orstructured data) on a computer-readable medium (e.g., as a hard disk; amemory; a computer network or cellular wireless network or other datatransmission medium; or a portable media article to be read by anappropriate drive or via an appropriate connection, such as a DVD orflash memory device) so as to enable or configure the computer-readablemedium and/or one or more associated computing systems or devices toexecute or otherwise use or provide the contents to perform at leastsome of the described techniques. Some or all of the components and/ordata structures may be stored on tangible, non-transitory storagemediums. Some or all of the system components and data structures mayalso be stored as data signals (e.g., by being encoded as part of acarrier wave or included as part of an analog or digital propagatedsignal) on a variety of computer-readable transmission mediums, whichare then transmitted, including across wireless-based andwired/cable-based mediums, and may take a variety of forms (e.g., aspart of a single or multiplexed analog signal, or as multiple discretedigital packets or frames). Such computer program products may also takeother forms in other embodiments. Accordingly, embodiments of thisdisclosure may be practiced with other computer system configurations.

The various embodiments described above can be combined to providefurther embodiments. All of the U.S. patents, U.S. patent applicationpublications, U.S. patent applications, foreign patents, foreign patentapplications and non-patent publications referred to in thisspecification and/or listed in the Application Data Sheet areincorporated herein by reference, in their entirety. Aspects of theembodiments can be modified, if necessary to employ concepts of thevarious patents, applications and publications to provide yet furtherembodiments.

While embodiments of the invention have been illustrated and described,as noted above, many changes can be made without departing from thespirit and scope of the invention. Accordingly, the scope of theinvention is not limited by the above disclosure.

1-20. (canceled)
 21. A method performed by a client device in a virtualpresence system, the method comprising: in a client module executing onthe client device, transmitting client data to a host module executingon a remote host device, wherein the client data includes motion dataand first position data that describes a location and/or orientationselected by a user of the client device; causing the remote host moduleto: select image data from a panoramic camera based on the firstposition data received from the client module, wherein the panoramiccamera is communicatively coupled to the remote host device; move thepanoramic camera or image capture orientation of the panoramic camera incorrelation with the received motion data; and transmit the selectedimage data to the client module; and presenting the received image dataon a display associated with the client device.
 22. The method of claim21, wherein presenting the received image data includes transmitting thereceived image data to a head-mounted display worn by the user, andfurther comprising: receiving from the head-mounted display the firstposition data.
 23. The method of claim 21, wherein the client device isa smart phone, and wherein the host device is a smart phone with anassociated panoramic camera.
 24. The method of claim 21, furthercomprising: transmitting audio and video data to the host module alongwith the position data; and receiving audio and video data from the hostmodule, such that the user of the client device communicates with one ormore users of the remote host device.
 25. The method of claim 21,further comprising: receiving second position data from the host module,wherein the second position data describes a location and/or orientationof the panoramic camera; and presenting along with the received imagedata an indication that the panoramic camera has moved.
 26. The methodof claim 21, further comprising: transmitting the selected image data toa viewing device, thereby causing the viewing device to present theselected image data.
 27. The method of claim 26, wherein the viewingdevice is a head-mounted display worn by the user, wherein thehead-mounted display is configured to transmit the first position datato the client module executing on the first mobile device.
 28. Anon-transitory computer-readable medium that includes instructions thatare configured, when executed by a computer processor in a clientdevice, to perform a method in a virtual presence system, the methodcomprising: in a client module executing on the client device,transmitting client data to a host module executing on a remote hostdevice, wherein the client data includes motion data and first positiondata that describes a location and/or orientation selected by a user ofthe client device; causing the remote host module to: select image datafrom a panoramic camera based on the first position data received fromthe client module, wherein the panoramic camera is communicativelycoupled to the remote host device; move the panoramic camera or imagecapture orientation of the panoramic camera in correlation with thereceived motion data; and transmit the selected image data to the clientmodule.
 29. The computer-readable medium of claim 28, wherein the remotehost module selects the image data by: selecting image data from aportion of a field of view of the panoramic camera, the portiondetermined based on the first position data, such that the selectedimage data reflects the location and/or orientation selected by theuser.
 30. The computer-readable medium of claim 28, wherein the remotehost module selects the image data by: receiving second position datathat describes a location and/or orientation of the panoramic camera;and selecting image data from the panoramic camera based on the firstposition data and the second position data, wherein the selected imagedata is obtained from a portion of a field of view of the panoramiccamera, the portion determined based on the first position data whilecompensating for camera movements reflected in the second image data.31. The computer-readable medium of claim 28, wherein the host deviceincludes multiple display devices, and wherein the method furtherincludes: receiving image data from the client module; and presentingthe image data on the multiple display devices.
 32. Thecomputer-readable medium of claim 28, wherein the host device includesan array of multiple microphones, and wherein the method furtherincludes: determining the position of a speaking user based on acousticsource localization performed on audio signals received by the multiplemicrophones; and transmitting an indication of the position of thespeaking user to the client module.
 33. A virtual telepresence system,comprising: a host device that includes a host module that isconfigured, when executed by the host device, to: receive client datafrom a client module executing on a remote client device, wherein theclient data includes motion data and first position data that describesa location and/or orientation selected by a user of the client device;move the panoramic camera or image capture orientation of the panoramiccamera in correlation with the received motion data; obtain image datafrom a panoramic camera based on the received first position data; andtransmit the obtained image data to the client module for presentationon a display associated with the client device.
 34. The system of claim33, wherein the host module obtains the image data by: selecting imagedata from a portion of a field of view of the panoramic camera, theportion determined based on the first position data, such that theselected image data reflects the location and/or orientation selected bythe user.
 35. The system of claim 33, wherein the host module obtainsthe image data by: receiving second position data that describes alocation and/or orientation of the panoramic camera; and selecting imagedata from the panoramic camera based on the first position data and thesecond position data, wherein the selected image data is obtained from aportion of a field of view of the panoramic camera, the portiondetermined based on the first position data while compensating forcamera movements reflected in the second image data.
 36. The system ofclaim 33, wherein the host device includes multiple display devices, andwherein the host module is further configured to: receive image datafrom the client module; and present the image data on the multipledisplay devices.
 37. The system of claim 33, wherein the host deviceincludes an array of multiple microphones, and wherein the host moduleis further configured to: determine the position of a speaking userbased on acoustic source localization performed on audio signalsreceived by the multiple microphones; and transmit an indication of theposition of the speaking user to the client module.
 38. The system ofclaim 33, wherein the host device is mounted on an unpiloted aerialvehicle.
 39. The system of claim 33, wherein the host device is mountedto a fixed structure.
 40. The system of claim 33, wherein the hostdevice is a video conferencing device.